Automated Teardown

The book has now been published and the content of this chapter has likely changed substanstially.
Please see page 503 of xUnit Test Patterns for the latest information.
Also known as: Test Object Registry

How do we tear down the Test Fixture?

We keep track of all resources that are created in a test and automatically destroy/free them during teardown.

Sketch Automated Teardown embedded from Automated Teardown.gif

A large part of making tests repeatable and robust is ensuring that the test fixture is torn down after each test. Leftover objects and database records, open files and connections can at best cause performance degradations and at worst cause tests to fail or systems to crash. While some of these resources may be cleaned up automatically by garbage collection, others may be left hanging if they are not torn down explicitly.

Writing tear down code that can be relied upon to clean up properly in all possible circumstances is challenging and time-consuming. It involves understanding what could be left over for each possible outcome of the test and writing code to deal with it. This Complex Teardown (see Obscure Test on page X) introduces a fair bit of Conditional Test Logic (page X) and worst of all Untestable Test Code (see Hard to Test Code on page X).

A better solution is to let the test infrastructure track the objects created and clean them up auto-magically when the test is complete.

How It Works

The core of the solution is a mechanism to register each persistent item (object, record, connection, etc.) we create in the test. We maintain a list (or lists) of registered objects that need some action to be taken to destroy them. This can be as simple as tossing each object into a collection. At the end of the test, we traverse the collection and destroy each object. We will want to catch any errors that we encounter so that one object's cleanup error will not cause the rest of the cleanup to be aborted.

When To Use It

We can use Automated Teardown whenever we have persistent resources that need to be cleaned up to keep the test environment functioning. (This happens more often in customer tests than in unit tests.) This pattern addresses Unrepeatable Test (see Erratic Test on page X) and Interacting Tests (see Erratic Test) by keeping objects created in one test from lingering on into the execution of a subsequent test.

It isn't very difficult to build and it will save us a large amount of grief and effort. Once we have built it for one project, we should be able to reuse it on subsequent projects for very little effort.

Implementation Notes

Automated Teardown comes in two flavors. The more basic flavor only tears down object created as paft of fixture setup. The more advanced version also destroys any objects that were created by the system under test (SUT).

Variation: Automated Fixture Teardown

The simplest solution is to register the objects we create in our Creation Methods (page X). This will not tear down the objects created by the SUT but by dealing with our fixture, it reduces the effort and likelihood of errors significantly.

The two key challenges are:

Finding generic way to cleanup the registered objects, and
ensuring that our Automated Teardown code is run for each.

Since the latter is the easier problem, let us deal with it first. When we are tearing down a Persistent Fresh Fixture (see Fresh Fixture on page X), the simplest solution is to put the call to the Automated Teardown mechanism into the tearDown method on our Testcase Class (page X). This method is called regardless of whether the test passes or fails as long as the setUp method succeeds. When we are tearing down a Shared Fixture (page X) we only want the method run after all the Test Methods (page X) have been run. We can user either SuiteFixture Setup (page X) if our member of the xUnit family supports or a Setup Decorator (page X).

Now back to the harder problem: the generic mechanism for cleaning up the resources. We have at least two options here. One is to ensure that all persistent (non-garbage-collected) objects implement a generic cleanup mechanism that we can call from within the Automated Teardown mechanism. The other alternative is to wrap each object in another object that knows how to clean up the object in question. The latter is an example of the Command[GOF] pattern.

If we build our Automated Teardown mechanism in a completely generic way, we can build it into the Testcase Superclass (page X) on which we can base all our Testcase Classes. Otherwise, we may need to put it onto a Test Helper (page X) that is visible from all the Testcase Classes that need it. A Test Helper that both creates fixture objects and tears them down automatically is sometimes called an Object Mother (see Test Helper).

Being a non-trivial (and very critical) piece of code, the Automated Teardown mechanism deserves its own unit test. Since it is now outside the Test Method, we can write Self-Checking Tests (see Goals of Test Automation on page X) for it! If we want to be really careful (some might say paranoid), we can use Delta Assertions (page X) to verify that any objects that exist after the tear down already existed before the test.

Variation: Automated Exercise Teardown

We can make the tests even more "self-cleaning" by also cleaning up the objects created by the SUT. This involves designing the SUT using an observable Object Factory (see Dependency Lookup on page X) so that we can automatically register any objects created by the SUT while it is being exercised. During the tear down phase we can delete these objects, too.

Motivating Example

In this example, we have created several objects using Creation Methods and need to tear them down when the test in complete. To do this, we have introduced a try/finally block to ensure that our Inline Teardown (page X) code is executed even when the assertions fail.

   public void testGetFlightsByOrigin_NoInboundFlight_SMRTD() throws Exception {
      // Fixture setup
      BigDecimal outboundAirport = createTestAirport("1OF");
      BigDecimal inboundAirport = null;
      FlightDto expFlightDto = null;
      try {
         inboundAirport = createTestAirport("1IF");
         expFlightDto = createTestFlight(outboundAirport, inboundAirport);
         // Exercise System
         List flightsAtDestination1 = facade.getFlightsByOriginAirport(inboundAirport);
         // Verify Outcome
         assertEquals(0,flightsAtDestination1.size());
      } finally {
         try {
            facade.removeFlight(expFlightDto.getFlightNumber());
         } finally {
            try {
               facade.removeAirport(inboundAirport);
            } finally  {
               facade.removeAirport(outboundAirport);
            } 
         }             
      }           
   }
Example SafeMultiResourceGuaranteedTeardown embedded from java/com/clrstream/ex6/services/test/InlineTeardownExampleTest.java

Note that we must use nested try/finally constructs within the finally block to ensure that any errors in the tear down don't keep us from finishing the job.

Refactoring Notes

Introducing Automated Teardown involves two steps. First, we add the Automated Teardown mechanism to our Testcase Class. Second, we remove any Inline Teardown code from our tests.

Automated Teardown can be implemented on a specific Testcase Class or it can be inherited (or mixed-in) via a generic class. Either way, we need to make sure we register all our newly-created objects so that the mechanism can know to delete them when the test is done. This is most easily done inside Creation Methods that already exist or we can use an Extract Method[Fowler] refactoring to remove direct constructor call into newly-created Creation Methods and add the registration.

The generic Automated Teardown mechanism should be invoked from the tearDown method. This can be done on our own Testcase Class but it is almost always better to put it on a Testcase Superclass that all our Testcase Classes inherit from. If we don't already have a Testcase Superclass we can easily create one by doing an Extract Class[Fowler] refactoring and then doing a Pull Up Method[Fowler] refactoring on any methods (and fields) associated with the mechanism.

Example: Automated Teardown

There is not much to see in this refactored test because all the tear down code code has been removed.

   public void testGetFlightsByOriginAirport_OneOutboundFlight()
   throws Exception {
      // Fixture setup
      BigDecimal outboundAirport = createTestAirport("1OF");   
      BigDecimal inboundAirport = createTestAirport("1IF");
      FlightDto expectedFlightDto = createTestFlight( outboundAirport, inboundAirport);
      // Exercise System
      List flightsAtOrigin = facade.getFlightsByOriginAirport(outboundAirport);
      // Verify Outcome
      assertOnly1FlightInDtoList( "Flights at origin", expectedFlightDto,
                                  flightsAtOrigin);
   }
Example JavaAutomatedTeardownTest embedded from java/com/clrstream/ex6/services/test/AutomatedTeardownExampleTest.java

Here is where all the work gets done! The Creation Methods has been modified to register the object it just created.

   private List allAirportIds;
   private List allFlights;
  
   protected void setUp() throws Exception {
      allAirportIds = new ArrayList();
      allFlights = new ArrayList();
   }  
  
   private BigDecimal createTestAirport(String airportName)
   throws FlightBookingException {
      BigDecimal newAirportId = facade.createAirport(
                       airportName, " Airport" + airportName,
                       "City" + airportName);
      allAirportIds.add(newAirportId);
      return newAirportId;
   }
Example JavaAutomatedTeardownRegistration embedded from java/com/clrstream/ex6/services/test/AutomatedTeardownExampleTest.java

Here is the actual Automated Teardown logic. In this example we are showing it living right on our Testcase Class and being called from the tearDown method. To keep this example very simple, it has been written specifically to handle airports and flights. More typically, it would live in the Testcase Superclass where it could be used by all our Testcase Classes and would use a generic object destruction mechanism so that it would care what types of objects it was deleting.

   protected void tearDown() throws Exception {
      removeObjects(allAirportIds, "Airport");
      removeObjects(allFlights, "Flight");
   }
  
   public void removeObjects(List objectsToDelete, String type) {
      Iterator i = objectsToDelete.iterator();
      while (i.hasNext()) {
         try {
            BigDecimal id = (BigDecimal) i.next();
            if ("Airport"==type) {         
               facade.removeAirport(id);
            } else {
               facade.removeFlight(id);
            }
         } catch (Exception e) {
            // do nothing if the remove failed
         }
      }
   }
Example JavaAutomatedTeardownMethod embedded from java/com/clrstream/ex6/services/test/AutomatedTeardownExampleTest.java

If we were tearing down a Shared Fixture, we would annotate our tearDown method with the suitable annotation or attribute (e.g. @afterClass or [TestFixtureTearDown]) or move it to a Setup Decorator.

Example: Automated Exercise Teardown

If we wanted to go the next step to automatically tear down any objects created within the SUT, we could modify the SUT to use an observable Object Factory and in our test we would add:

ResourceTracker tracker;

public void setUp() {
   tracker = new ResourceTracker();
   ObjectFactory.addObserver(tracker);
}

public void tearDown() {
   tracker.cleanup();
   ObjectFactory.removeObserver(tracker);
}
Example JavaAutomatedExerciseTeardown embedded from Fixture Teardown/fixtureTeardown.txt

This last example assumes that the Automated Teardown logic has been moved into the cleanup method on the ResourceTracker.

Page generated at Wed Feb 09 16:39:36 +1100 2011